Perception and action are inseparable

In her target article, Michaels defines action as “a temporally bounded, observable, goaldirected movement (or non-movement) that entails intention, the detection of information, and a lawful relation between that information and the movement”. She defines perception as “the detection of information”. This forces one to conclude that it is impossible to study action separately from perception. We argue that perceptual judgements are communicated by movements, and that it is impossible to distinguish movements reporting perceptual judgements from other movements. So we conclude that also the reverse is true: it is impossible to study perception separately from action. Accepted for publication in Ecological Psychology Smeets & Brenner Perception and action are inseparable 2 Before trying to distinguish between perception and action, one must of course define them accurately. Michaels starts by defining perception (ecologically) as “the detection of information” (p7). She points out that a clear definition of action is missing in the work of Goodale and Milner (1992), and comes with a (new) definition of action: “a temporally bounded, observable, goal-directed movement (or non-movement) that entails intention, the detection of information, and a lawful relation between that information and the movement” (p15). This definition binds action to perception, as it states that there can be no action without detection of information (which is perception). Can there be perception without action? The problem with this question is the measurement of perception. There is no way to do so other than by activating muscles to speak, press a button, move a mouse, etc. If one would accept that these movements are actions, one could conclude that there is no measurable perception without action. Various authors claim that movements which report perceptual judgement are fundamentally different from “real actions”. Michaels' definition of action tries to capture the intuitive idea that pressing a button to report a perceptual judgement is not a real action. In order to write this paper, I press long sequences of buttons on my keyboard. Is typing an action? It is definitely intentional (I want to write the paper), it entails the detection of information (I monitor and correct errors), and there is a lawful relation between the information and the movement (when I see a “w” missing, my left index-finger moves to the position of the “w” on the upper left side of my keyboard). All three criteria of an action are met: typing is an action. One might argue that having to move towards the upper left side of the keyboard when a “w” is missing is an arbitrary rather than a lawful relationship, because the position of the “w" depends on the country you are in (QWERTY or AZERTY keyboard).What constitutes a lawful relation? A clear (very strict) example of a lawful relation between the information and the movement is an “isomorphism” (Bridgeman & Huemer, 1998). If one requires an isomorphism, typing cannot be considered an action anymore, but is a perceptual report. At a lower level, however, typing still consists of actions. After having chosen the right key, the movement towards it is based on an isomorphism. Thus perceptual reports are conveyed by actions. An example of this entanglement between perception and action is the experiment of Aglioti, DeSouza, and Goodale (1995) mentioned in the target article. In their experiment, subjects viewed two disks, and had to grasp the left one if it was the largest (or smallest). The direction in which subjects started to move (i.e. their choice) was not isomorphic with the perceptual information on size, and the authors consequently consider it to reflect a perceptual judgement. The opening of their hand and the forces used to grasp it are isomorphic with the Smeets & Brenner Perception and action are inseparable 3 size of the disk, and can thus be considered as (aspects of an) action. Thus different aspects of a single grasping movement can be considered either perception or action. The key to the distinction is the isomorphism (or some other lawful relationship). The fundamental difficulty with lawful relationships is that they are not fixed. The relation between the missing “w” and where my finger has to move depends on the kind of keyboard used. For grasping a disk, the exact relationships both between its size and the forces used to lift it, and between its size and the opening of the hand depend on various factors (such as the material the disk is made of and movement speed; Smeets & Brenner, 1999). Moreover, the opening of the hand would be independent of disk size if one’s task was to push the largest disk away rather than to pick it up. So one could claim that the task determines what is lawful. Probably, this is a way to circumvent “intention”, a term which is used in Michaels’ definition of action, but is itself not defined. Our conclusion is that perception and action are not the right categories, neither to discuss information processing in the brain (for alternative categories see: Ungerleider and Haxby, 1994), nor to solve issues in (ecological) psychology. We think that differences between tasks or aspects of tasks in for example the influence of illusions (e.g. Aglioti et al., 1995) are related to differences in the information used. The effect of size-contrast illusions on perceptual judgements depends strongly on the exact format of the display (Pavani, Boscagli, Benvenuti, Rabuffetti, and Farnè, 1999; Franz, Gegenfurtner, Bülthoff, and Fahle, 2000). Such illusions can also affect one size-related aspect of an action (e.g. the forces to lift it), without influencing another size-related aspect of the same action (Smeets & Brenner, 1996; Jackson & Shaw, 2000). Differences between tasks also clearly influence the way we pick up information. How we move our eyes when we look at a picture depends on what perceptual judgement we are asked to make (Yarbus, 1967, p. 174). How our head and eyes are coordinated (the example mentioned in the target article) depends on the task: it is not only important whether the hand moves or not, but also what the hand is doing (Smeets, Hayhoe and Ballard, 1996). So the distinction is not between perception and action, but between the information used for various aspects of a task, independent of whether the task is considered “perception” or “action”. Why then does an illusion influence one aspect of a task while another aspect of the same task remains uninfluenced? The answer is that various aspects of a task are based on different sources of information. We can illustrate this with an experiment we did a few years ago, in which we asked subjects to hit running spiders (Smeets & Brenner, 1995). A moving background influenced perceived speed but not perceived position. When hitting moving targets, the background influenced the hand's maximum speed, but not its initial direction. Both these two aspects of a hitting movement are mathematically isomorphic with the speed Smeets & Brenner Perception and action are inseparable 4 of the target. According to Bridgeman & Huemer (1998), both aspects are thus part of an action. Therefore, the differential effects of the illusion on these variables cannot be explained by a distinction between perception and action. We argued that these findings could be explained by assuming that the hand's initial direction is based on information on the (perceived) position of the target, independent of its speed, whereas the maximum speed of the hand is based the (perceived) speed of the target. Using this assumption, we could also explain the differences in the shape of the trajectories towards targets moving at different speeds (Smeets & Brenner, 1995). In this example two sources of information are involved: the speed and the (changing) position of the spider. In the real world, it is physically impossible to change speed without changing positions in a corresponding way. An illusion, however, can do this trick. Look for instance at some motion (e.g. a waterfall) for a minute or so, and then look at a stationary object (e.g. a tree). The result is that you see the tree moving upwards, without changing its position. Illusions of speed will influence any aspect of interception that depends on velocity, but not one that depends on (change in) position (Smeets & Brenner, 1995). Similarly, illusions of size will influence aspects of grasping like grip force that depend on size, but not aspects like grip aperture that depend on positions (Smeets & Brenner, 1999). In this way, one can describe the information used for each aspect of a task. We see no advantage in trying to define two separate categories (perception and action) to classify all aspects of human behaviour.